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(57) Abstract 



An object detection system including passive sensors (3) for receiving electromagnetic radiation from a moving object (28) and 
generating intensity signals representative of the received radiation, and a processing system for subtracting the intensity signals to obtain 
a differential signature representative of the position of the moving object. An image acquisition system including at least one camera (7) 
for acquiring an image of at least part of a moving object, in response to a trigger signal, and an analysis system for processing the image 
to locate a region in the image including markings identifying the object and processing the region to extract the markings for optical 
recognition. 





Codes used to identify 


AL 
AM 


Albania 
Armenia 


AT 


Austria 


AU 


Australia 


AZ 


Azerbaijan 


BA 


Bosnia and Hetxegovma 


BB 


Barbados 


BE 


Be 


BF 


Bt iso 


BG 


Bu 


BJ 


Bei 


BR 


Braxi: 


BY 


Be Una 


CA 


Cauda 


CF 


Central African Republic 


CG 


Congo 


CH 


Switzerland 


a 


COte d'Woire 


CM 


Cameroon 


CN 


China. 


CU 


Cuba 


cz 


Cxech Republic 


DK 


Germany 


DK 


Denmark 



FOR THE PURPOSES OP INFORMATION ONLY 

i of pamphlets publishing international applications under the PCT. 



Fl 

FB 

GA 

GB 

GB 

GH 

GN 

GR 

HU 

IB 

1L 

IS 

IT 

JP 

KE 

KG 



KR 

KZ 

IX 

U 

LK 

LR 



Spain 
Finland 
France 
Gabon 

United Kingdom 



Greco 

Hunga 

Irelaoo 

Urael 

Iceland 

Italy 

Japan 

Kenya 

Kyrgyzstan 

Democratic People's 

Republic of Korea 

Republic of Korea 



Saint Lucia 
Liechtenstein 
Sri I 



LS 

LT 

LU 

LV 

MC 

MD 

MG 

MK 

ML 

MN 

MR 

MW 

MX 

NE 

NL 

NO 

NZ 

PL 

FT 

RO 

RU 



SG 



Lesotho 



Luxembourg 

Latvia 

Monaco 

Republic of Moldova 

Madagascar 

The former Yugoslav 

Republic of Macedonia 

Mali 

Mongolia 



Mexico 
Niger 

Netherlands 
Norway 
New Zealand 
Poland 
Portugal 



Russian Federation 



Sweden 
Singapore 



SI 


Slovenia 


SK 


Slovakia 


SN 


Senegal 


sz 


Swaziland 


TO 


Chad 


TG 


Togo 


TJ 


Tajikistan 


TM 


Turkmenistan 


TR 


Turkey 


TT 


Trinidad and Tobago 


UA 


Ukraine 


UG 


Uganda 


US 


United States of America 


UZ 


Uzbekistan 


VN 


Viet Nam 


YU 


Yugoslavia 


zw 


Zimbabwe 



WO 97/37336 



PCT/AU97/00198 



5 AN AIRCRAFT DETECTION SYSTEM 

The present invention relates to an object detection system and, in particular 
to an aircraft detection system. 

1 0 The International Civil Aviation Organisation (ICAO) has established regulations 

which require all civil aircraft to have registration markings beneath the port wing to 
identify an aircraft. The markings denote the nationality of an aircraft and its 
registration code granted by the ICAO. In some countries, airline operators do not 
follow the regulations and the markings appear on an aircraft's fuselage. Owners of 

15 aircraft are charged for airport use, but a satisfactory system has not been developed 
to automatically detect aircraft and then, if necessary, administer a charge to the 
owner. Microwave signals for detecting an aircraft can interfere with microwave 
frequencies used for airport communications and, similarly, radar signals can interfere 
with those used for aircraft guidance systems. A system which can be used to detect 

20 an aircraft using unobtrusive passive technology is desired. 

In accordance with the present invention there is provided an object detection 
system including: 

passive sensing means for receiving electromagnetic radiation from a moving 
25 object and generating intensity signals representative of the received radiation; and 
processing means for subtracting said intensity signals to obtain a differential 
signature representative of the position of said moving object. 

The present invention also provides an image acquisition system including: 
30 at least one camera for acquiring an image of at least part of a moving object, 

in response to a trigger signal, and 
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analysis means for processing said image to locate a region in said image 
including markings identifying said object and processing said region to extract said 
markings for a recognition process. 

5 The present invention also provides an object detection method including: 

passively sensing electromagnetic radiation received from a moving object; 
generating intensity signals representative of the received radiation; and 
subtracting said intensity signals to obtain a differential signature representative 
of the position of said moving object. 

10 

The present invention also provides an image acquisition method including: 
acquiring an image of at least part of a moving object, in response to a trigger 

signal, using at least one camera, and 

processing said image to locate a region in said image including markings 
15 identifying said object and processing said region to extract said markings for a 

recognition process. 

Preferred embodiments of the present invention are hereinafter described, by 
way of example only, with reference to the accompanying drawings, wherein: 
20 Figure 1 is a block diagram of a preferred embodiment of an aircraft detection 

system; 

Figure 2 is a schematic diagram of a preferred embodiment of the aircraft 
detection system; 

Figure 3 is a block diagram of a connection arrangement for components of the 

25 aircraft detection system; 

Figure 4 is a more detailed block diagram oi a proximity detector and a tracking 

system for the aircraft detection system, 

Figure 5 is a coordinate system used for the proximity detector; 

Figures 6(a) and 6(b) are underneath views of discs of sensors of the tracking 

30 system, 

Figure 7 is a schematic diagram of an image obtained by the tracking system; 
Figures 8 and 9 are images obtained from a first embodiment of the tracking 
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system; 

Figure 10 is a graph of a pixel row sum profile for an image obtained by the 
tracking system; 

Figure 11 is a graph of a difference profile obtained by subtracting successive 
5 row sum profiles; 

Figure 12 is a diagram of a coordinate system for images obtained by the 
tracking system; 

Figure 1 3 is a diagram of a coordinate system for the aircraft used for geometric 
correction of the images obtained by the tracking system; 
10 Figure 14 is a diagram of a coordinate system used for predicting a time to 

generate an acquisition signal; 

Figure 15 is a graph of aircraft position in images obtained by the tracking 
system over successive frames; 

Figure 16 is a graph of predicted trigger frame number over successive image 
15 frames obtained by the tracking system; 

Figure 17 is a schematic diagram of a pyroelectric sensor used in a second 
embodiment of the tracking system; 

Figure 18 is graphs of differential signatures obtained using the second 
embodiment of the tracking system; 
20 Figures 19 and 20 are images obtained of an aircraft by high resolution 

cameras of an acquisition system of the aircraft detection system; 

Figure 21 is a schematic diagram of an optical sensor system used for exposure 
control of the acquisition cameras; 

Figure 22 is a flow diagram of a preferred character location process executed 
25 on image data obtained by the high resolution cameras; 

Figure 23 is a diagram of images produced during the character location 
process; and 

Figure 24 is a flow diagram of a character recognition process executed on a 
binary image of the characters extracted from an image obtained by one of the high 
30 resolution cameras. 
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An aircraft detection system 2, as shown in Figure 1 , includes a proximity 
detector 4, a tracking sensor system 6, an image processing system 8, an image 
acquisition system 10 and an analysis system 12. A control system 14 can be included 
to control the image acquisition system 10 on the basis of signals provided by the 
5 image processing system 8, and also control an illumination unit 16. 

The proximity detector 4 and the tracking sensor system 6 includes sensors 3 
which may be placed on or near an aircraft runway 5 to detect the presence of an 
aircraft 28 using visual or thermal imaging or aural sensing techniques. Also located 

10 on or near the runway 5 is at least one high resolution camera 7 of the image 
acquisition system 10. The sensors 3 and the acquisition camera 7 are connected by 
data and power lines 9 to an instrument rack 1 1 , as shown in Figure 2, which may be 
located adjacent or near the runway 5. The instrument rack 1 1 may alternatively be 
powered by its own independent supply which may be charged by solar power . The 

15 instrument rack 1 1 includes control circuitry and image processing circuitry which is 
able to control activation of the sensors 3 and the camera 7 and perform image 
processing, as required. The instrument rack 1 1 , the data and power lines 9, the 
sensors 3 and the acquisition camera 7 can be considered to form a runway module 
which may be located at the end of each runway of an airport. A runway module can 

20 be connected back to a central control system 13 using an optical fibre or other data 
link 15. Images provided by the sensors 3 may be processed and passed to the central 
system 13 for further processing, and the central system 13 would control triggering 
of the acquisition cameras 7. Alternatively image processing for determining triggering 
of the acquisition camera 7 may be performed by each instrument rack 1 1 . The central 

25 control system 13 includes the analysis system 12. One method of configuring 
connection of the instrument racks 11 to the central control system 13 is illustrated in 
Figure 3. The optical fibre link 15 may include dedicated optical fibres 17 for 
transmitting video signals to the central control system 13 and other optical fibres 19 
dedicated to transmitting data to and receiving data from the central control system 1 3 
30 using the Ethernet protocol or direct serial data communication. A number of different 
alternatives can be used for connecting the runway modules to the central control 
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system 1 3. For example, the runway modules and the control system 1 3 may be 
connected as a Wide Area Network (WAN) using Asynchronous Transfer Mode (ATM) 
or Synchronous Digital Hierarchy (SDH) links. The runway modules and the central 
control system 13 may also be connected as a Local Area Network (LAN) using a LAN 
5 protocol, such as Ethernet. Physical connections may be made between the runway 
modules and the central control system 13 or alternatively wireless transmission 
techniques may be used, such as using infrared or microwave signals for 
communication. 

10 The proximity detector 4 determines when an aircraft is within a predetermined 

region, and then on detecting the presence of an aircraft activates the tracking sensor 
system 6. The proximity detector 4, as shown in Figure 4, may include one or more 
pyroelectric devices 21, judiciously located at an airport, and a signal processing unit 
23 and trigger unit 25 connected thereto in order to generate an activation signal to 

15 the tracking sensor system 6 when the thermal emission of an approaching aircraft 
exceeds a predetermined threshold. The proximity detector 4 may use one or more 
pyroelectric point sensors that detect the infrared radiation emitted from the aircraft 28. 
A mirror system can be employed with a point sensor 70 to enhance its sensitivity to 
the motion of the aircraft 28. The point sensor 70 may consist of two or more 

20 pyroelectric sensors configured in a geometry and with appropriate electrical 
connections so as to be insensitive to the background infrared radiation and slowly 
moving objects. With these sensors the rate of motion of the image of the aircraft 28 
across the sensor 70 is important. The focal length of the mirror 72 is chosen to 
optimise the motion of the image across the sensor 70 at the time of detection. As an 

25 example, if the aircraft at altitude H with glide slope angle 6 GS moves with velocity V 
and passes overhead at time f 0 , as shown in Figure 5, then the position h of the 
image of the aircraft 28 on the sensor 70 is 




where f is the focal length of a cylindrical mirror. 
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If the rate of motion of the image dhldt is required to have a known value, then 
the focal length of the mirror 72 should be chosen to satisfy 

= v «o - Q 2 dh (2) 
H dt 

where t 0 - f is the time difference between the time t 0 at which the aircraft is 
overhead and the time f at which it is to be detected. Alternatively, the proximity 
5 detector 4 may include different angled point sensors to determine when an aircraft 
enters the monitored region and is about to land or take-off. In response to the 
activation signal, the tracking sensor system 6 exposes the sensor 3 to track the 
aircraft. Use of the proximity detector 4 allows the sensor 3 to be sealed in a housing 
when not in use and protected from damaging environmental conditions, such as 
10 hailstorms and blizzards or fuel. The sensor 3 is only exposed to the environment for 
a short duration whilst an aircraft is in the vicinity of the sensor 3. If the tracking 
system 6 is used in conditions where the sensor 3 can be permanently exposed to the 
environment or the sensor 3 can resist the operating conditions, then the proximity 
detector 4 may not be required. The activation signal generated by the proximity 
1 5 detector 4 can also be used to cause the instrument rack 1 1 and the central control 
system 13 to adjust the bandwidth allocated on the link 15 so as to provide an 
adequate data transfer rate for transmission of video signals from the runway module 
to the central system 13. If the bandwidth is fixed at an acceptable rate or the system 
2 only uses local area network communications and only requires a reduced 
20 bandwidth, then again the proximity detector 4 may not be required. 

The tracking sensor system 6 includes one or more tracking or detection 
cameras 3 which obtain images of an aircraft as it approaches or leaves a runway. 
From a simple image of the aircraft, aspect ratios, such as the ratio of the wingspan 
25 to the fuselage length can be obtained. The tracking camera 3 used is a thermal 
camera which monitors thermal radiation received in the 10 to 14 M m wavelength 
range and is not dependent on lighting conditions for satisfactory operation. Use of the 
thermal cameras is also advantageous as distribution of temperatures over the 
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observed surfaces of an aircraft can be obtained, together with signatures of engine 
exhaust emissions and features in the fuselage or engines. The tracking camera 3 can 
obtain an instantaneous two-dimensional image l n using all of the sensors in a CCD 
array of the camera, or alternatively one row of the array perpendicular to the direction 
5 of motion of the aircraft can be used to obtain a linear image at each scan and the 
linear image is then used to build up a two-dimensional image l n for subsequent 
processing. 

To allow operation of the tracking and acquisition cameras 3 and 7 in rain, a 
1 0 rotating disc system is employed. The use of a rotating disc for removing water drops 
from windows is used on marine vessels. A reflective or transparent disc is rotated at 
high speed in front of the window that is to be kept clear. Water droplets falling on the 
disk experience a large shear force related to the rotation velocity. The shear force is 
sufficient to atomise the water drop, thereby removing it from the surface of the disc. 
1 5 A transparent disc of approximate diameter 200 mm is mounted to an electric motor 
and rotated to a frequency of 60 Hz. A camera with a 4.8 mm focal length lens was 
placed below a glass window which in turn was beneath the rotating disc. The results 
of inserting the rotating disc are illustrated in Figure 6(a), which shows the surface of 
a camera housing without the rotating disc, and in Figure 6(b), which shows the 
20 surface of a camera housing with the rotating disc activated and in rain conditions. 

The image processing system 8 processes the digital images provided by the 
tracking sensor system 6 so as to extract in real-time information concerning the 
features and movement of the aircraft. The images provided to the image processing 

25 system, depending on the tracking cameras employed, provide an underneath view 
of the aircraft, as shown in Figure 7. The tips of the wings or wingspan points 1 8 of the 
aircraft are tracked by the image processor system 8 to determine when the image 
acquisition system 10 should be activated so as to obtain the best image of the 
registration markings on the port wing 20 of the aircraft. The image processing system 

30 8 generates an acquisition signal using a trigger logic circuit 39 to trigger the camera 
of the image acquisition system 10. The image processing system 8 also determines 
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and stores data concerning the wingspan 22 of the aircraft and other details 
concerning the size, shape and ICAO category (A to G) of the aircraft. The image 
processing system 8 classifies the aircraft on the basis of the size which can be used 
subsequently when determining the registration markings on the port wing 20. The 
5 data obtained can also be used for evaluation of the aircraft during landing and/or 
take-off. 

Alternatively a pyroelectric sensor 27 can be used with a signal processing wing 
detection unit 29 to provide a tracking system 1 which also generates the acquisition 
10 signal using the trigger logic circuit 39, as shown in Figure 4 and described later. 

Detecting moving aircraft in the field of view of the sensor 3 or 27 is based on 
forming a profile or signature of the aircraft. P(y,f), that depends on a spatial 
coordinate y and time t . To eliminate features in the field of view that are secondary 

15 or slowly moving, a difference profile AP(y,f) is formed. The profile or signature can be 
differenced in time or in space because these differences are equivalent for moving 
objects, tf the intensity of the light or thermal radiation from the object is not changing 
then the time derivative of the profile obtained from this radiation is zero. A time 
derivative of a moving field can be written as a convective derivative involving partial 

20 derivatives, which gives the equation 

dPjy.t) _ dP(y,t) + v dP(y,t) = 0 (3) 
dt dt By 

where v is the speed of the object as observed in the profile. After rearranging 
equation (3), gives 

BP(y,t) = _ v §£M (4) 
dt dy 

which shows that the difference in the profile in time is equivalent to the difference in 
the profile in space. This only holds for moving objects, when v * 0. Equation (4) also 
25 follows from the simple fact that if the profile has a given valueP(y 0 ,g at the 
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coordinate (y 0 ,f 0 ), then it will have this same value along the line 

y = Yo + v(t - t 0 ) (5) 

To detect and locate a moving feature that forms an extremum in the profile, 
such as an aircraft wing, the profile can be differenced in space A^y.f). Then an 
extremum in the profile P(y, t) will correspond to a point where the difference profile A^y.f) 
5 crosses zero. 

In one method for detecting a feature on the aircraft, a profile P(y,f) is formed 
and a difference profile A/^y.f) is obtained by differencing in time, as described 
below. According to equation (4) this is equivalent to a profile of a moving object that 
10 is differenced in space. Therefore the position y p of the zero crossing point of A,P(y,f) 
at time t is also the position of the zero crossing point of A y P(y t t) which locates an 
extremum in P(y,t). 

In another method for detecting a feature on the aircraft, the difference between 
15 the radiation received by a sensor 27 from two points in space is obtained as a 
function of time, A y S(t) , as described below. If there are no moving features in the field 
of view, then the difference is constant. If any object in the field of view is moving, then 
the position of a point on the object is related to time using equation (5). This allows 
a profile or signature differenced in space to be constructed 

A/W),0 = A y S(/) (6) 

20 and, as described above, allows an extremum corresponding to an aircraft wing to be 
located in the profile from the zero crossing point in the differential signature. 

The image acquisition system 10 includes at least one high resolution camera 
7 to obtain images of the aircraft when triggered. The images are of sufficient 
25 resolution to enable automatic character recognition of the registration code on the 
port wing 20 or elsewhere. The illumination unit 16 is also triggered simultaneously to 
provide illumination of the aircraft during adverse lighting conditions, such as at night 
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or during inclement weather. 

The acquired images are passed to the analysis system 12 which performs 
Optical Character Recognition (OCR) on the images to obtain the registration code. 
5 The registration code corresponds to aircraft type and therefore the aircraft 
classification determined by the image processing system 8 can be used to assist to 
the recognition process, particularly when characters of the code are obscured in an 
acquired image. The registration code extracted and any other information concerning 
the aircraft can be then passed to other systems via a network connection 24. 

10 

Once signals received from the pyroelectric sensors 21 indicate the aircraft 28 
is within the field of view of the sensors 3 of the tracking sensor system 6, the tracking 
system 1 is activated by the proximity detector 4. The proximity detector 4 is usually 
the first stage detection system to determine when the aircraft is in the proximity of the 
15 more precise tracking system 1. The tracking system 1 includes the tracking sensor 
system 6 and the image processing system 8 and according to one embodiment the 
images from the detection cameras 3 of the sensor system 6 are used by the image 
processing system 8 to provide a trigger for the image acquisition system when some 
point in the image of the aircraft reaches a predetermined pixel position. One or more 
20 detection cameras 3 are placed in appropriate locations near the airport runway such 
that the aircraft passes within the field of view of the cameras 3. A tracking camera 3 
provides a sequence of images, {/„}. The image processing system 8 subtracts a 
background image from each image /„of the sequence. The background image 
represents an average of a number of preceding images. This yields an image A/„ that 
25 contains only those objects that have moved during the time interval between images. 
The image A/„ is thresholded at appropriate values to yield a binary image, i.e. one 
that contains only two levels of brightness, such that the pixels comprising the edges 
of the aircraft are clearly distinguishable. The pixels at the extremes of the aircraft in 
the direction perpendicular to the motion of the aircraft will correspond to the edges 
30 18 of the wings of the aircraft. After further processing, described below, when it is 
determined the pixels comprising the port edge pass a certain position in the image 
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corresponding to the acquisition point, the acquisition system 10 is triggered, thereby 
obtaining an image of the registration code beneath the wing 20 of the aircraft. 

Imaging the aircraft using thermal infrared wavelengths and detecting the 
5 aircraft by its thermal radiation renders the aircraft self-luminous so that it can be 
imaged both during the day and night primarily without supplementary illumination. 
Infrared (IR) detectors are classified as either photon detectors (termed cooled 
sensors herein), or thermal detectors (termed uncooled sensors herein). Photon 
detectors (photoconductors or photodiodes) produce an electrical response directly 

10 as the result of absorbing IR radiation. These detectors are very sensitive, but are 
subject to noise due to ambient operating temperatures. It is usually necessary to 
cryogenically cool (80° K) these detectors to maintain high sensitivity. Thermal 
detectors experience a temperature change when they absorb IR radiation, and an 
electrical response results from temperature dependence of the material property. 

15 Thermal detectors are not generally as sensitive as photon detectors, but perform well 
at room temperature. 

Typically, the cooled sensing devices are formed from Mercury Cadmium 
Telluride offer far greater sensitivity than uncooled devices, which may be formed from 

20 Barium Strontium Titanate. Their Net Equivalent Temperature Difference (NETD) is 
also superior. However, with the uncooled sensor a chopper can be used to provide 
temporal modulation of the scene. This permits AC coupling of the output of each pixel 
to remove the average background. This minimises the dynamic range requirements 
for the processing electronics and amplifies only the temperature differences. This is 

25 an advantage for resolving differences between cloud, the sun, the aircraft and the 
background. The advantage of differentiation between objects is that it reduced the 
load oh subsequent image processing tasks for segmenting the aircraft from the 
background and other moving objects such as the clouds. 

30 Both a cooled and uncooled thermal infrared imaging system 6 has been used 

during day, night and foggy conditions. The system 6 produced consistent images of 
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the aircraft in all these conditions, as shown in Figures 8 and 9. In particular, the sun 
in the field of view produced no saturation artefacts or flaring in the lens. At night, the 
entire aircraft was observable, not just the lights. 

5 The image processing system 8 uses a background subtraction method in an 

attempt to eliminate slowly moving or stationary objects from the image, leaving only 
the fast moving objects. This is achieved by maintaining a background image that is 
updated after a certain time interval elapses. The update is an incremental one based 
on the difference between the current image and the background. The incremental 

10 change is such that the background image can adapt to small intensity variations in 
the scene but takes some time to respond to large variations. The background image 
is subtracted from the current image, a modulus is taken and a threshold applied. The 
result is a binary image containing only those differences from the background that 
exceed the threshold. 

15 

One problem with this method is that some slow moving features, such as 
clouds, still appear in the binary image. The reason for this is that the method does not 
select on velocity but on a combination of velocity and intensity gradients. If the 
intensity in the image is represented by /(x.y.f), where x and y represent the position 
20 in rows and columns, respectively, and t represents the image frame number (time) 
and if the variation in the intensity due to ambient conditions is very small then it can 
be shown that the time variation of the intensity in the image due to a feature moving 
with velocity v is given by 

a/(x ' y - f) = -v.V/(x.y,f) (7) 
dt 

In practice, the time derivative in equation (7) is performed by taking the 
25 difference between the intensity at (x,y) at different times. Equation (7) shows that the 
value of this difference depends on the velocity v of the feature at (x.y) and the 
intensity gradient. Thus a fast moving feature with low contrast relative to the 
background is identical to a slow moving feature with a large contrast. This is the 
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situation with slowly moving clouds that often have very bright edges and therefore 
large intensity gradients there, and are not eliminated by this method. Since features 
in a binary image have the same intensity gradients, better velocity selection is 
obtained using the same method but applied to the binary image. In this sense, the 
5 background-subtraction method is applied twice, once to the original grey-scale image 
to produce a binary image as described above, and again to the subsequent binary 
image, as described below. 

The output from the initial image processing hardware is a binary image B(x,y,f) 
10 where B(x,y,t) = 1 if a feature is located at (x,y) at time t , and S(x,y,f) = Orepresents 
the background. Within this image the fast moving features belong to the aircraft. To 
deduce the aircraft wing position the two-dimensional binary image can be 
compressed into one dimension by summing along each pixel row of the binary image, 

P(y,f) = f B(x,y,f) dx (8) 

where the aircraft image moves in the direction of the image columns. This row-sum 
15 profile is easily analysed in real time to determine the location of the aircraft. An 
example of a profile is shown in Figure 10 where the two peaks 30 and 31 of the 
aircraft profile correspond to the main wings (large peak 30) and the tail wings (smaller 
peak 31 ). 

20 In general, there are other features present, such as clouds, that must be 

identified or filtered from the profile. To do this, differences between profiles from 
successive frames are taken, which is equivalent to a time derivative of the profile. 
Letting A(x,y y t) be the aircraft where A(x,y t t) = 1 if (x,y) lies within the aircraft and 0 
otherwise and letting C(x t yJ) represent clouds or other slowly moving objects, then 

25 it can be shown that the time derivative of the profile is given by 

= [i&M (1 - C (x,y,0]*x ♦ e(C) (9) 

J of 
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where e(C) - 0 is a small error term due to the small velocity of the clouds. Equation 
(9) demonstrates an obvious fact that the time derivative of a profile gives information 
on the changes (such as motion) of feature A only when the changes in A do not 
overlap features C. In order to obtain the best measure of the location of a feature, the 

5 overlap between features must be minimised. This means that C(x,y,f) must cover as 
small an area as possible. If the clouds are present but do not overlap the aircraft, 
then apart from a small error term, the time difference between profiles gives the 
motion of the aircraft. The difference profile corresponding to Figure 10 is shown in 
Figure 11 where the slow moving clouds have been eliminated. The wing positions 

10 occur at the zero-crossing points 33 and 34. Note that the clouds have been removed, 
apart from small error terms. 

The method is implemented using a programmable logic circuit of the image 
processing system 8 which is programmed to perform the row sums on the binary 
1 5 image and to output these as a set of integers after each video field. When taking the - 
difference between successive profiles the best results were obtained using 
differences between like fields of the video image, i.e. even-even and odd-odd fields. 

The difference profile is analysed to locate valid zero crossing points 
20 corresponding to the aircraft wing positions. A valid zero crossing is one in which the 
difference profile initially rises above a threshold l T for a minimum distance y T and 
falls through zero to below -l T for a minimum distance y T . The magnitude of the 
threshold l T is chosen to be greater than the error term e(C) which is done to discount 
the affect produced by slow moving features, such as clouds. 

25 

In addition, the peak value of the profile, corresponding to the aircraft wing, can 
be obtained by summing the difference values when they are valid up to the zero 
crossing point. This method removes the contributions to the peak from the non- 
overlapping clouds. It can be used as a guide to the wing span of the aircraft. 

30 

The changes in position of the aircraft in the row-sum profile are used to 
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determine a velocity for the aircraft that can be used for determining the image 
acquisition or trigger time, even if the aircraft is not in view. This situation may occur 
if the aircraft image moves into a region on the sensor that is saturated, or if the tngger 
point is not in the field of view of the camera 3. However, to obtain a reliable estimate 
5 of the velocity, geometric corrections to the aircraft position are required to account 
for the distortions in the image introduced by the camera lens. These are descnbed 
below using the coordinate systems (x.y.z) for the image and (X, Y,Z) for the aircraft 
as shown in Figures 12 and 13, respectively. 

1 o For an aircraft at distance Z and at a constant altitude Y 0 , the angle from the 

horizontal to the aircraft in the vertical plane is 

tan6 y = | < 10 > 

Since Y 0 is approximately constant, a normalised variableZ N = ZIY 0 can be used. If y 0 
is the coordinate of the centre of the images, f is the focal length of the lens and 9 C 
is the angle of the camera from the horizontal in the vertical plane, then 

yo-y _ tanfe -eu tan9 y- tan9 - dD 

__ - tan(O y e c ) 1 + ^ ^ 

1 5 where the tangent has been expanded using a standard trigonometric identity. Using 
(10) and (11) an expression for the normalised distance Z w is obtained 

_ 1 ; P(y - y 0 >an9 c (12) 
' tan6 e - (ty - y 0 ) 

where p = Mf. This equation allows a point in the image at y to be mapped onto a 
true distance scale, Z N . Since the aircraft altitude is unknown, the actual distance 
cannot be determined. Instead, all points in the image profile are scaled to be 
20 equivalent to a specific point, y, . in the profile. This point is chosen to be the trigger 
line or image acquisition line. The change in the normalised distance ZJyJ at y, due 
to an increment in pixel value Ay, is AZ^y,) = Z„(y, + Ay,) - Z^y,). The number 



WO 97/37336 



PCT/AU97/00198 



-16- 



of such increments over a distance Z^y 2 ) - Z N (yJ is M = (Z^y 2 ) - Z^y^/AZ^y,). 
Thus the geometrically corrected pixel position at y 2 is 

*- - * * ^ ■ * * z ^f'' ««» 

For an aircraft at distance Z and at altitude Y 0 , a length X on it in the X direction 
subtends an angle in the horizontal plane of 

tan e x = = (14) 

5 where normalised values have been used. If x 0 is the location of the centre of the 
image and f is the focal length of the lens, then 

!LL!!2. = tane, (15) 



Using (12), (14) and (15), the normalised distance X N can be obtained in terms of x 
and y 

X N - L-^ftT^fo) (16) 

As with the y coordinate, the x coordinate is corrected to a value at y, . Since X N 
10 should be independent of position, then a length x 2 - x 0 at y 2 has a geometrically 
corrected length of 



, 1 * 

X c2 X 0 " \ X 2 " X l 



v'i * z^(y t ) (17) 



v </i * p 2 (k 2 - s|nQ c - tyi "j^ 39 ' 

= " Xo) sin6 c - ty 2 - y 0 )cos8 c ^ 7 p^, - ytf 

The parameter p = 1/f is chosen so that x and y are measured in terms of pixel 
numbers. If y 0 is the centre of the camera centre and it is equal to half the total 
number of pixels, and if 8 FOV is the vertical field of view of the camera, then 
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p = _JLf°!Li (18) 

This relation allows p to be calculated without knowing the lens focal length and the 
dimensions of the sensor pixels. 

The velocity of a feature is expressed in terms of the number of pixels moved 
5 between image fields (or frames). Then if the position of the feature in frame n is y„, 
the velocity is given by v n = y n - y n ^ . Over N frames, the average velocity is then 

«-i| v "-3£fc ,19) 

which depends only on the start and finish points of the data. This is sensitive to errors 
in the first and last values and takes no account of the positions in between. The error 
in the velocity due to an error 6y N in the value y N is 

««*» = ^ (20) 

10 A better method of velocity estimation uses all the position data obtained between 
these values. A time t is maintained which represents the cunrent frame number. Then 
the current position is given by 

y = y 0 - vf (21) 

where y 0 is the unknown starting point and v is the unknown velocity. The number n 
of valid positions y n measured from the feature are each measured at time t n . 
15 Minimising the mean square error 

1 N 

X 2 = -E(K„ - y 0 + vfjf (22) 



with respect to v and y 0 gives two equations for the unknown quantitiesy 0 and v. 
Solving for v yields 
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V = 



N N N 



(23) 



"E'n-E'nE'* 



This solution is more robust in the sense that it takes account of all the motions of the 
feature, rather than the positions at the beginning and at the end of the observations. 
If the time is sequential, so that t n = nbt where t„ = 1 is the time interval between 
image frames, then the error in the velocity due to an error 6y„ in the value y„ is 



5 which, for the same error by N in (19), gives a smaller error than (21) for N > 5. In 
general, the error in (24) varies as MN 2 which is less sensitive to uncertainties in 
position than (19). 

If the aircraft is not in view, then the measurement of the velocity v can be used 
1 0 to estimate the trigger time. If y, is the position of a feature on the aircraft that was last 
seen at a time t,, then the position at any time t is estimated from 



Based on this estimate of position, the aircraft will cross the trigger point located 
at y r at a time t T estimated by 



An alternative method of processing the images obtained by the camera 3 to 
15 determine the aircraft position, which also automatically accounts for geometric 
corrections, is described below. The method is able to predict the time for triggering 
the acquisition system 10 based on observations of the position of the aircraft 28. 




(24) 



y = y,-v(t- f ( ) 



(25) 
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To describe the location of an aircraft 28 and its position, a set of coordinates 
are defined such that the x axis points vertically upwards, the z axis points 
horizontally along the runway towards the approaching aircraft, and j/ is horizontal 
and perpendicular to the runway. The image 66 of the aircraft is located in the digitised 
5 image by pixel values (x p ,y p ), where x p is defined to be the vertical pixel value and y p 
the horizontal value. The lens on the camera inverts the image so that a light ray from 
the aircraft strikes the sensor at position (-x p ,-y pl 0), where the sensor is located at 
the coordinate origin. Figure 14 shows a ray 68 from an object, such as a point on the 
aircraft, passing through a lens of a focal length f , and striking the imaging sensor at 
10 a point (-x pi -y p ), where x p andy p are the pixel values. The equation locating a point 
on the ray is given by 

r = x p {zlf - 1)* c + y p (zlf - W c + zi c {2 7) 

where z is the horizontal distance along the ray, and the subscript c refers to the 
camera coordinates. The camera axis i c is collinear with the lens optical axis. It will 
be assumed that z/f»1 , which is usually the case. 

15 

Assuming the camera is aligned so that y c = y is aligned with the runway 
coordinate, but the camera is tilted from the horizontal by angle 6. Then 

x c = x cos9 - z sine 

i c = x sin8 + 2 cos6 < 28 > 

and a point on the ray from the aircraft to its image is given by 
r = z[(x p cos8 / f + sin8) x + (y p / /) y + (cos8 - x p sine / /) i] (29) 

Letting the aircraft trajectory be given by 

KO = (z(t) tan9 GS - x 0 ) x + Yo y + z(0z (30) 

20 where z(t) is the horizontal position of the aircraft at time t , 6 GS is the glide-slope 
angle, and the aircraft is at altitude x 0 and has a lateral displacement y 0 at z(t 0 ) = 0. 
Here, t = t Q is the time at which the image acquisition system 10 is triggered, i.e. when 
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the aircraft is overhead with respect to the cameras 7. 

Comparing equations (29) and (30) allows z to be written in terms of z(f) and 
gives the pixel positions as 



x D (0 = f 



' z(f)[cos8 tan6 GS - sine] + x 0 cosO ^ 



^ z(f)[sin9 tan6 GS + cos8] + x 0 sine ; 



(31) 



5 and 



v (f) = — ; « (32) 

p z(f)[sin6 tan9 GS + cos8] ♦ x 0 sinS 

Since z p (t) is the vertical coordinate and its value controls the acquisition 
trigger, the following discussion will be centred on equation (31 ) The aircraft position 
is given by 

z(f) = v(t 0 - t) (33) 
where v is the speed of the aircraft along the 1 axis. 

10 

The aim is to determine f 0 from a series of values of z p (f) at t determined from 
the image of the aircraft. For this purpose, it is useful to rearrange (31) into the 
following form 

c - f + ax p - btx p = 0 (34) 

where 

g vf 0 (tan8 GS * cote) + x 0 (35) 
9 ~ ft^l - tan8 GS cot8) 

tan8 GS , cot8 
/(1 - tan6 GS cote) 



15 and 
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0 " M1 - tan6 GS cote) (37) 

The pixel value corresponding to the trigger point vertically upwards is x r = f cot9 . 
The trigger time, f 0 , can be expressed in terms of the parameters a, b and c 

c + ax T 

The parameters a, b and c are unknown since the aircraft glide slope, speed, 
altitude and the time at which the trigger is to occur are unknown. However, it is 
5 possible to estimate these using equation (34) by minimising the chi-square statistic. 
Essentially, equation (34) is a prediction of the relationship between the measured 
values x p and t , based on a simple model of the optical system of the detection 
camera 3 and the trajectory of the aircraft 28. The parameters a, b and c are to be 
chosen so as to minimise the error of the model fit to the data, i.e. make equation (34) 
10 be as close to zero as possible. 

Let x n be the location of the aircraft in the image, i.e. pixel value, obtained at 
time t n . Then the chi-square statistic is 

X 2 - E (c - t n + ax„ - btjc n f (39) 

for N pairs of data points. The optimum values of the parameters are those that 
15 minimise the chi-square statistic, i.e. those that satisfy equation (34). 



For convenience, the following symbols are defined 

N N N N 

/>=1 n = 1 

N N N 

= T.x't R = Fx/ 2 s = Yxhl 
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Then the values of a, b and c that minimise equation (39) are given by 



a 



(NP - XljP 2 - NS) - (NR - PT)(PX - NQ) 
(A/Y - X 2 \P 2 - NS) * (PX - NQ) 2 "~" 



(41) 



(A/P - XTjPX - A/Q) ♦ (Atf? - PVliNY - X 2 ) 
(A/Y - X 2 \P 2 - NS) + (PX - NQ) 2 



(42) 



and 



c = 



T * bP - aX 
N 



(43) 



On obtaining a, b and c from equations (41) to (43), then t 0 can be obtained from 
equation (38). 



Using data obtained from video images of an aircraft landing at Melbourne 
airport, a graph of aircraft image position as a function of image frame number is 
shown in Figure 15. The data was processed using equations (41 ) to (43) and (38) to 
yield the predicted value for the trigger frame number f 0 = 66 corresponding to trigger 

1 0 point 70. The predicted point 70 is shown in Figure 16 as a function of frame number. 
The predicted value is f 0 = 66 ± 0.5 after 34 frames. In this example, the aircraft can 
be out of the view of the camera 3 for up to 1 .4 seconds and the system 2 can still 
trigger the acquisition camera 7 to within 40 milliseconds of the correct time. For an 
aircraft travelling at 62.5 m/s, the system 2 captures the aircraft to within 2.5 metres 

1 5 of the required position. 

The tracking system 6, 8 may also use an Area-Parameter Accelerator (APA) 
digital processing unit, as discussed in International Publication No. WO 93/19441, 
to extract additional information, such as the aspect ratio of the wing span to the 
20 fuselage length of the aircraft and the location of the centre of the aircraft. 

The tracking system 1 can also be implemented using one or more pyroelectric 
sensors 27 with a signal processing wing detection unit 29. Each sensor 27 has two 



5 
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adjacent pyroelectric sensing elements 40 and 42, as shown in Figure 17, which are 
electrically connected so as to cancel identical signals generated by each element. A 
plate 44 with a slit 46 is placed above the sensing elements 40 and 42 so as to 
provide the elements 40 and 42 with different fields of view 48 and 50. The fields of 
5 view 48 and 50 are significantly narrower than the field of view of a detection camera 
discussed previously. If aircraft move above the runway in the direction indicated by 
the arrow 48, the first element 40 has a front field of view 48 and the second element 
42 has a rear field of view 50. As an aircraft 28 passes over the sensor 27 the first 
element 40 detects the thermal radiation of the aircraft before the second element 42, 

1 0 the aircraft 28 will then be momentarily in both fields of view 48 and 50, and then only 
detectable by the second element 42. An example of the difference signals generated 
by two sensors 27 is illustrated in Figure 18 where the graph 52 is for a sensor 27 
which has a field of view that is directed at 90° to the horizontal and a sensor 27 which 
is directed at 75° to the horizontal Graph 54 is an expanded view of the centre of 

1 5 graph 52. The zero crossing points' of peaks 56 in the graphs 52 and 54 correspond 
to the point at which the aircraft 28 passes the sensor 27. Using the known position 
of the sensor 27 the time at which the aircraft passes, and the speed of the aircraft 28, 
a time can be determined for generating an acquisition signal to trigger the high 
resolution acquisition cameras 7. The speed can be determined from movement of the 

20 zero crossing points over time, in a similar manner to that described previously. 

The image acquisition system 10, as mentioned previously, acquires an image 
of the aircraft with sufficient resolution for the aircraft registration characters to be 
obtained using optical character recognition. According to one embodiment of the 
25 acquisition system 10, the system 10 includes two high resolution cameras 7 each 
comprising a lens and a CCD detector array. Respective images obtained by the two 
cameras 7 are shown in Figures 19 and 20. 

The minimum pixel dimension and the focal length of the lens determine the 
30 spatial resolution in the image. If the dimension of a pixel is L p , the focal length f and 
the altitude of the aircraft is /?, then the dimension of a feature 14/ . on the aircraft that 
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is mapped onto a pixel is 

W . = or f = (44) 

mm f W 

1 rr min 

The character recognition process used requires each character stroke to be 
mapped onto at least four pixels with contrast levels having at least 10% difference 
from the background. The width of a character stroke in the aircraft registration is 

5 regulated by the ICAO. According to the ICAO Report, Annex 7, sections 4.2.1 and 
5.3, the character height beneath the port wing must be at least 50 centimetres and 
the character stroke must be 1/6 1 " the character height. Therefore, to satisfy the 
character recognition criterion, the dimension of the feature on the aircraft that is 
mapped onto a pixel should be W min = 2 centimetres, or less. Once the CCD detector 

10 is chosen, L p is fixed and the focal length of the system 10 is determined by the 
maximum altitude of the aircraft at which the spatial resolution W min = 2 centimetres 
is required. 

The field of view of the system 10 at altitude h is determined by the spatial 
1 5 resolution W min chosen at altitude h max and the number of pixels Np, along the length 
of the CCD, 

W F0V = * min (45) 

"max 

For h = and = 1552 the field of view is W F0V = 31.04 metres. 

To avoid blurring due to motion of the aircraft, the image moves a distance less 
20 than the size of a pixel during the exposure. If the aircraft velocity is v, then the time 
to move a distance equal to the required spatial resolution is 

W 

t = -J™! (46) 
v 



The maximum aircraft velocity that is likely to be encountered on landing or 
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take-off is v = 160 knots = 82 ms" 1 . With W mjn = 0.02 m, the exposure time to 
avoid excessive blurring is f < 240 iu.s . 

The focal length of the lens in the system 1 0 can be chosen to obtain the 
5 required spatial resolution at the maximum altitude. This fixes the field of view. 
Alternatively, the field of view may be varied by altering the focal length according to 
the altitude of the aircraft. The range of focal lengths required can be calculated from 
equation (44). 

10 The aircraft registration, during daylight conditions, is illuminated by sunlight 

or scattered light reflected from the ground. The aircraft scatters the light that is 
incident, some of which is captured by the lens of the imaging system. The 
considerable amount of light reflected from aluminium fuselages of an aircraft can 
affect the image obtained, and is taken into account. The light power falling onto a 

1 5 pixel of the CCD is given by 

P - / AAO R w* R * A p fA -.\ 
r p - L x AAU sun — — — (47) 

where L K is the solar spectral radiance, AA is the wavelength bandpass of the entire 
configuration, O sun is the solid angle subtended by the sun, R gnd is the reflectivity of 
the ground, R A is the reflectivity of the aircraft, A p is the area of a pixel in the CCD 
detector and /# is the lens f-number. 

20 

The solar spectral radiance L A varies markedly with wavelength A. The power 
failing on a pixel will therefore vary over a large range. This can be limited by 
restricting the wavelength range AA passing to the sensor and optimally choosing the 
centre wavelength of this range. The optimum range and centre wavelength are 
25 chosen to match the characteristics of the imaging sensor. 

In one embodiment, the optimum wavelength range and centre wavelength are 
chosen in the near infrared waveband, 0.69 to 2.0 microns. This limits the variation in 
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light power on a pixel in the sensor to within the useable limits of the sensor. A 
KODAK™ KAF-1600L imaging sensor (a monolithic silicon sensor with lateral overflow 
anti-blooming) was chosen that incorporated a mechanism to accommodate a 
thousandfold saturation of each pixel, giving a total acceptable range of light powers 
5 in each pixel of 10 s . This enables the sensor to produce a useful image of an aircraft 
when very bright light sources, for example the sun, are in its field of view. 

The correct choice of sensor and the correct choice of wavelength range and 
centre wavelength enables an image to be obtained within a time interval that arrests 
10 the motion of the aircraft and that provides an image with sufficient contrast on the 
aircraft registration to enable digital image processing and recognition of the 
registration characters. 

In choosing the wavelength range and centre wavelength, it was important to 
1 5 avoid dazzling light from the supplementary illumination of the illumination unit 1 6. The 
optimum wavelength range was therefore set to between 0.69 and 2.0 ^m. 

The power of sunlight falling onto a pixel directly from the sun is 

p = LU^F- (48) 

The relative light powers from the sun and from the aircraft registration falling 
20 onto a single pixel is 

Pp ^sun R gnd R A 

With = 6.8 * 10 s steradians, - 0.2 and R A = 1 , the ratio is 
P IP = 4 6 x 1 0 s This provides an estimate of the relative contrast between the 

r p-sun' p r 

image of the sun and the image of the underneath of the aircraft on a CCD pixel. The 
CCD sensor and system electronics are chosen to accommodate this range of light 
25 powers. 
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In poor lighting conditions, the aircraft registration requires additional 
illumination from the illumination unit 16. The light source of the unit 16 needs to be 
sufficient to illuminate the aircraft at its maximum altitude. If the source is designed to 
emit light into a solid angle that just covers the field of view of the imaging system then 
5 the light power incident onto a pixel of the imaging system 1 0 due to light emitted from 
the source and reflected from the aircraft is given by 

R A 

p ' * p -*z ^ m 

where A A is the area on the aircraft imaged onto a pixel of area A p , P s is the light 
power of the source, R A is the aircraft reflectivity, A/ ptot is the total number of pixels in 
the CCD sensor and f# is the f-number of the lens. The power of the source required 

10 to match the daytime reflected illumination is estimated by setting 
P p = 7.3 x 10 11 W, R A = 1, A p = 81 M m 2 , N m = 1552 * 1032, /# = 1.8 and 
noting that /\ A = W*^ where W min = 0.02 m.Then P s = 1.50 * 10 4 W. For a Xenon 
flash lamp, the flash time is typically t = 300 /zs which compares favourably with the 
exposure time to minimise motion blurring. Then the source must deliver an energy of 

15 £ s = P s t = 4.5 J . This is the light energy in a wavelength band AA = 0.1 /zm. Xenon 
flash lamps typically have 10% of their light energy within this bandpass centred 
around A = 0.8 ^m. Furthermore, the flash lamp may only be 50% efficient. Thus the 
electrical energy required is approximately 90 J. Flash lamps that deliver energies of 
1500 J in 300 ms are readily available. Illumination with such a flash lamp during the 

20 day reduces the contrast between the direct sun and the aircraft registration, thereby 
relaxing the requirement for over-exposure tolerance of the CCD sensor. This result 
depends on the flash lamp directing all of its energy into the field of view only and that 
the lens focal length is optimally chosen to image the region of dimension 
^min = 0 02 roonto a single pixel. 

25 

In one embodiment, the aperture of the lens on the acquisition camera 7 is 
automatically adjusted to control the amount of light on the imaging sensor in order to 
optimise the image quality for digital processing. In the image obtained, the intensity 
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level of the registration characters relative to the underside of the aircraft needs to be 
maintained to provide good contrast between the two for OCR. The power P s of the 
flash 16 is automatically adjusted in accordance with the aperture setting /# of the 
acquisition camera 7 to optimise the image quality and maintain the relative contrast 

5 between the registration characters and the underside of the aircraft, in accordance 
with the relationship expressed in equation (50). For example, during the day the 
aperture of the lens may be very small and the power of the flash may be increased 
to provide additional illumination of the underside, whereas during night conditions, 
the aperture may be fully opened and the power of the flash reduced considerably as 

10 additional illumination is not required. As an alternative, or in addition, the electrical 
gain of the electronic circuits connected to an acquisition camera 7 is adjusted 
automatically to optimise the image quality. 

To appropriately set the camera aperture and/or gain one or more point optical 
15 sensors 60, 62, as shown in Figure 21, are used to measure the ambient lighting 
conditions. The electrical output signals of the sensors 60, 62 are processed by the 
acquisition system 10 to produce the information required to control the camera 
aperture and/or gain. Two point sensors 60, 62 sensitive to the same optical spectrum 
as the acquisition cameras 7 can be used. One sensor 60 receives light from the sky 
20 that passes through a diffusing plate 64 onto the sensor 60. The diffusing plate 64 
collects light from many different directions and allows it to reach the sensor 60. The 
second sensor 62 is directed towards the ground to measure the reflected light from 
the ground. 



The high resolution images obtained of the aircraft by the acquisition system 
10 are submitted, as described previously, to the analysis system 12 which performs 
optical character recognition on the images to extract the registration codes of the 
aircraft. 

The analysis system 12 processes the aircraft images obtained by a high 
resolution camera 7 according to an image processing procedure 100, as shown in 
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Figure 22, which is divided into two parts 102 and 104. The first part 102 operates on 
a sub-sampled image 1 05, as shown in Figure 23, to locate regions that contain 
features that may be registration characters, whereas the second part 104 executes 
a similar procedure but is done using the full resolution of the original image and is 
5 executed only on the regions identified by the first part 102. The sub-sampled image 
105 is the original image with one pixel in four removed in both row and column 
directions, resulting in a one in sixteen sampling ratio. Use of the sub-sampled image 
improves processing time sixteen-fold. 

10 The first part 102 receives the sub-sampled image at step 106 and filters the 

image to remove features which are larger than the expected size of the registration 
characters (b) at step 108. Step 108 executes a morphological operation of linear 
closings applied to a set of lines angled between 0 and 180°. The operation passes 
a kernel or window across the image 105 to extract lines which exceed a 

15 predetermined length and are at a predetermined angle. The kernel or window is 
passed over the image a number of times and each time the predetermined angle is 
varied. The lines extracted from all of the passes are then subtracted from the image 
105 to provide a filtered difference image 109. The filtered difference image 109 is 
then thresholded or binarised at step 1 10 to convert it from a grey scale image to a 

20 binary scale image 111. This is done by setting to 1 all image values that are greater 
than a threshold and setting to 0 all other image values. The threshold at a given point 
in the image is determined from a specified multiple of standard deviations from the 
mean calculated from the pixel values within a window centred on the given point. The 
binarised image 1 1 1 is then filtered at step 1 12 to remove all features that have pixel 

25 densities in a bounding box that are smaller or larger than the expected pixel density 
for a bounded registration character. The image 1 1 1 is then processed at step 1 14 to 
remove all features which are not clustered together like registration characters. Step 
114 achieves this by grouping together features that have similar sizes and that are 
close to one another. Groups of features that are smaller than a specified size are 

30 removed from the image to obtain a cleaned image 113. The cleaned image 1 13 is 
then used at step 116 to locate regions of interest. Regions of interest are obtained 



WO 97/37336 



PCT/AU97/00198 



-30- 

in step 1 16 from the location and extents of the groups remaining after step 114. Step 
1 16 produces regions of interest which include the registration characters and areas 
of the regions are bounded above and below, as for the region 115 shown in 
Figure 23. 

5 

The regions of interest obtained by the first part 102 of the procedure 100 are 
further processed individually using the full resolution of the original image and the 
second part 104 of the procedure. The second part 1 04 takes a region of interest 1 1 5 
from the original image at step 120 and for that region filters out features larger than 

10 the expected character sizes at step 122, using the same morphological operation of 
linear closings applied to a set of lines angled between 0 and 180°, followed by image 
subtraction, as described above, to obtain image 117. The filtered image 1 17 is then 
binarised at step 124 by selecting a filter threshold that is representative of the pixel 
values at the edges of features. To distinguish the registration characters from the 

15 aircraft wing or body the filter threshold needs to be set correctly. A mask image of 
significant edges in image 1 17 is created by calculating edge-strengths at each point 
in image 117 and setting to 1 all points that have edge-strengths greater than a mask 
threshold and setting to 0 all other points. An edge-strength is determined by taking 
at each point pixel gradients in two directions, Ax and Ay, and calculating 

20 y'Ax 2 + Ay 2 to give the edge-strength at that point. The mask threshold at a given 
point is determined from a specified multiple of standard deviations from the mean 
calculated from the edge-strengths within a window centred on the given point. The 
filter threshold for each point in image 1 17 is then determined from a specified multiple 
of standard deviations from the mean calculated from the pixel values at all points 

25 within a window centred on the given point that correspond to non-zero values in the 
mask image. The binarised image 118 is then filtered at step 126 to remove features 
that are smaller than the expected character sizes. Features are clustered together at 
step 128 that have similar sizes, that are near to one another and that are associated 
with similar image values in image 117. At step 130 the correctly clustered features 
30 that have sizes, orientations and relative positions that deviate too much from the 
averages for the clusters are filtered out to leave features that form linear chains. Then 
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at step 132, if the number of features remaining in the image produced by step 130 is 
greater than 3, then a final image is created by rotating image 1 18 to align the linear 
chain of features with the image rows and by masking out features not belonging to 
the linear chain. The final image is passed to a character recognition process 200 to 
5 determine whether the features are registration characters and, if so, which 
characters. 

The final image undergoes a standard optical character recognition process 
200, as shown in Figure 24, to generate character string data which represents the 

1 0 ICAO characters on the port wing. The process 200 includes receiving the final image 
at step 202, which is produced by step 132 of the image processing procedure 100, 
and separating the characters of the image at step 204. The size of the characters are 
normalised at step 206 and at step 208 correction for the alignment of the characters 
is made and further normalisation occurs. Character features are extracted at step 210 

1 5 and an attempt made to classify the features of the characters extracted at step 21 2. 
Character rules are applied to the classified features at step 214 so as to produce a 
binary string representative of the registration characters at step 216, 

Although the system 2 has been described above as being one which is 
20 particularly suitable for detecting an aircraft, it should be noted that many features of 
the system can be used for detecting and identifying other moving objects. For 
example, the embodiments of the tracking system 1 may be used for tracking land 
vehicles. The system 2 may be employed to acquire images of and identify 
automobiles at tollway points on a roadway. 

25 

Many modifications will be apparent to those skilled in the art without departing 
from the scope of the present invention as hereinbefore described with reference to 
the accompanying drawings. 
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CLAIMS: 

1 . An object detection system including: 

passive sensing means for receiving electromagnetic radiation from a moving 
5 object and generating intensity signals representative of the received radiation; and 
processing means for subtracting said intensity signals to obtain a differential 
signature representative of the position of said moving object 

2. An object detection system as claimed in claim 1, wherein said processing 
10 means generates an image acquisition signal on the basis of said differential 

signature. 

3. An object detection system as claimed in claim 2, including acquisition means, 
responsive to said acquisition signal, for acquiring an image of at least part of said 

15 moving object; and 

analysis means for processing said image to identify said moving object. 

4. An object detection system as claimed in claim 3, wherein said analysis system 
processes said acquired image to extract markings to identify said moving object. 

20 

5. An object detection system as claimed in claim 1 , 2, 3 or 4, wherein said moving 
object is an aircraft. 

6. An object detection system as claimed in claim 5, wherein said aircraft is in 
25 flight. 

7. An object detection system as claimed in claim 5, wherein said electromagnetic 
radiation is thermal radiation. 
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8. An object detection system as claimed in claim 5, including proximity detecting 
means for detecting the presence of said aircraft within a predetermined region and, 
in response thereto, generating an activation signal for said passive sensing means. 

5 9. An object detection system as claimed in claim 1, wherein said passive sensing 
means includes imaging means for obtaining images of said radiation at successive 
periods of time, and 

said processing means generates respective profiles of pixel values for said 
images and a difference profile, generated from the difference between successive 
10 profiles, which includes said differential signature. 

10. An object detection system as claimed in claim 9, wherein said position 
corresponds to a zero crossing point in said difference profile where said difference 
profile has risen above a first predetermined threshold for at least a first predetermined 

15 distance and then falls to below a second predetermined threshold for a second 
predetermined distance. 

11. An object detection system as claimed in claim 10, wherein said processing 
means monitors the movement of said position in successive ones of said difference 

20 profile and determines the time for generation of an image acquisition signal. 

12. An object detection system as claimed in claim 11, wherein said processing 
means generates a background image from successive images obtained by said 
imaging means and subtracts said background image from images of said radiation 

25 before generating said profiles. 

1 3. An object detection system as claimed in claim 1 , wherein said passive sensing 
means includes pyroelectric sensors with different fields of view and said intensity 
signals include at least first and second signals representative of the thermal radiation 

30 present in said views, respectively, and 
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said processing means subtracts said first and second signals to obtain a 
differential signal including said differential signature. 

14. An object detection system as claimed in claim 13, wherein said processing 
5 means determines a time for generation of an image acquisition signal on the basis 

of the position of said passive sensing means, the time of generation of said 
differential signature and the speed of said moving object. 

15. An object detection system as claimed in claim 14, wherein said time of 
10 generation and said speed are determined from a zero crossing point of said 

differential signature. 

1 6. An image acquisition system including: 

at least one camera for acquiring an image of at least part of a moving object, 
15 in response to a trigger signal, and 

analysis means for processing said image to locate a region in said image 
including markings identifying said object and processing said region to extract said 
markings for a recognition process. 

20 17. An image acquisition system as claimed in claim 16, wherein said camera 
images received radiation between 0.69 to 2.0 ^m. 

1 8. An image acquisition system as claimed in claim 17, wherein said camera has 
an exposure time of < 240 ^s 

25 

19. An image acquisition system as claimed in claim 1 7, including an infrared flash 
having its power adjusted on the basis of the aperture setting of said camera. 

20. An image acquisition system as claimed in claim 17, including optical sensor 
30 means positioned to obtain measurements of ambient direct light and reflected light 

for the field of view of said camera and adjust settings of said camera on the basis of 
said measurements. 



WO 97/37336 



PCT/AU97/00198 



-35- 

21. An image acquisition system as claimed in claim 16, wherein said analysis 
means sub-samples said image, extracts lines exceeding a predetermined length and 
at predetermined angles, binarises the image, removes features smaller or larger than 
said markings, removes features not clustered as said markings, and locates said 

5 region using the remaining features. 

22. An image acquisition system as claimed in claim 21, wherein said analysis 
means extracts said region from said image and processes said region by removing 
features larger than expected marking sizes, binarising said region, removing features 

10 smaller than expected marking sizes, removing features not clustered as identifying 
markings, and passing the remaining image for optical recognition if including more 
than a predetermined number of markings. 

23. An image acquisition system as claimed in claim 17, wherein said moving object 
15 is an aircraft. 

24. An image acquisition system as claimed in claim 23, wherein said aircraft is in 
flight. 

20 25. An object detection system as claimed in claim 2, including an image acquisition 
system as claimed in any one of claims 16 to 24, wherein said trigger signal is said 
image acquisition signal. 

26. An object detection method including: 
25 passively sensing electromagnetic radiation received from a moving object; 

generating intensity signals representative of the received radiation; and 
subtracting said intensity signals to obtain a differential signature representative 

of the position of said moving object. 

30 27. An object detection method as claimed in claim 26, including generating an 
image acquisition signal on the basis of said differential signature. 
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28. An object detection method as claimed in claim 27, including acquiring an 
image of at least part of said moving object in response to said acquisition signal; and 
processing said image to identify said moving object. 

5 29. An object detection method as claimed in claim 28, including processing said 
acquired image to extract markings identifying said moving object. 

30. An object detection method as claimed in claim 26, 27, 28 or 29, wherein said 
moving object is an aircraft. 

10 

31 . An object detection method as claimed in claim 30. wherein said aircraft is in 
flight. 

32. An object detection method as claimed in claim 30, wherein said 
15 electromagnetic radiation is thermal radiation. 

33. An object detection method as claimed in claim 30, including detecting the 
presence of said aircraft within a predetermined region and, in response thereto, 
generating an activation signal to execute said passive sensing step. 

20 

34. An object detection method as claimed in claim 26, wherein said passive 
sensing includes imaging said radiation at successive periods of time, and 

said subtracting includes generating respective profiles of pixel values for 
images of said radiation and generating a difference profile, from the difference 
25 between successive profiles, which includes said differential signature. 

35. An object detection method as claimed in claim 34, wherein said position 
corresponds to a zero crossing point in said difference profile where said difference 
profile has risen above a first predetermined threshold for at least a first predetermined 

30 distance and then falls to below a second predetermined threshold for a second 
predetermined distance. 
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36. An object detection method as claimed in claim 35, including monitoring the 
movement of said position in successive ones of said difference profile and 
determining the time for generation of an image acquisition signal. 

5 37. An object detection method as claimed in claim 36, wherein said subtracting 
includes generating a background image from successive images of said radiation 
imaging means and subtracting said background image from images of said radiation 
before generating said profiles. 

10 38. An object detection method as claimed in claim 26, wherein said passive 
sensing includes pyroelectric sensing with different fields of view and said intensity 
signals include at least first and second signals representative of the thermal radiation 
present in said views, respectively, and 

said subtracting includes subtracting said first and second signals to obtain a 

15 differential signal including said differential signature. 

39. An object detection method as claimed in claim 38, including determining a time 
for generation of an image acquisition signal on the basis of the position of passive 
sensors, the time of generation of said differential signature and the speed of said 

20 moving object. 

40. An object detection method as claimed in claim 39, wherein said time of 
generation and speed are determined from a zero crossing point of said differential 
signature. 

25 

41 . An image acquisition method including: 

acquiring an image of at least part of a moving object, in response to a trigger 
signal, using at least one camera, and 

processing said image to locate a region in said image including markings 
30 identifying said object and processing said region to extract said markings for a 
recognition process. 
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42. An image acquisition method as claimed in claim 41, wherein said camera 
images received radiation between 0.69 to 2.0 ^m. 

43. An image acquisition method as claimed in claim 42, wherein said camera has 
5 an exposure time of < 240 us. 

44. An image acquisition method as claimed in claim 42, including adjusting the 
power of an infrared flash for said camera on the basis of the aperture setting of said 
camera. 

10 

45. An image acquisition method as claimed in claim 42, including obtaining 
automatic measurements of ambient direct light and reflected light for the field of view 
of said camera and adjusting settings of said camera on the basis of said 
measurements. 

15 

46. An image acquisition method as claimed in claim 41, wherein said image 
processing includes sub-sampling said image, extracting lines exceeding a 
predetermined length and at predetermined angles, binarising the image, removing 
features smaller or larger than said markings, removing features not clustered as said 

20 markings, and locating said region using the remaining features. 

47. An image acquisition method as claimed in claim 46, wherein said region 
processing includes extracting said region from said image, removing features larger 
than expected marking sizes, binarising said region, removing features with areas 

25 smaller or larger than expected marking areas, removing features not clusterec as 
identifying markings, and passing the remaining image for optical recognition if 
including more than a predetermined number of markings. 



48. An image acquisition method as claimed in claim 42, wherein said moving 
30 object is an aircraft. 
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49. An image acquisition method as claimed in claim 48, wherein said aircraft is in 
flight. 

50. An object detection method as claimed in claim 27, including an image 
5 acquisition method as claimed in any one of claims 41 to 49, wherein said trigger 

signal is said image acquisition signal. 
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